Here's What I Think We Should Do

A Proactive AI Policy Agenda

For the last two months, I have been working on an essay entitled “Artificial Intelligence and American Rebirth: Self-Governance at the Frontier.” It turned into something more like a short book. I am still deciding how, and whether, I wish to release this longer piece. But in the meantime, I think it is time to get the key ideas out there. The essay below is partially adapted from the book-essay-manifesto project. Over the coming weeks, I will publish a few other adaptations.

Introduction

The fog is clearing. We can start to make out structures in the distance. Inch by inch, the contours of our future are coming into view.

Machines with the ability to match human intellect, and sometimes wildly exceed it, will be built within a few years. For an ever-increasing range of cognitive tasks, they are already here. They will only become more potent, and they will do so at a rapid pace. They are likely to outthink even the smartest among us in at least some important domains.

AI should be understood primarily as a discovery rather than an invention, as an empirical fact about nature that humans have, through monumental effort, uncovered. We have discovered how to make matter think.

Thinking is, at least in part, the ability to solve problems, and in so doing to discover new knowledge. It will allow us to automate substantial parts of the process of invention itself. It is therefore more appropriate to think not just of the effects of AI per se, but also the many technological and scientific breakthroughs it will enable, culminating in a broad range of new inventions. This blossoming of “transformative AI” is likely to feel more like the beginning of a new period, a new epoch in history, than a discrete event.

Unsurprisingly, there is a broad range of opinions about what government should do with respect to AI. But how do we regulate an industrial revolution? How do we regulate an era?

There is no way to pass “a law,” or a set of laws, to control an industrial revolution. That is not what laws are for. Laws are the rules of the game, not the game itself. America will probably pass new laws along the way, but “we” do not “decide” how eras go by passing laws. History is not some highway with “guardrails.” Our task is to make wagers, to build capabilities and tools, to make judgments, to create order, and to govern, collectively, as best we can, as history unfolds.

In most important ways, America is better positioned than any other country on Earth to thrive amid the industrial revolution to come. To the extent AI is a race, it is ours to lose. To the extent AI is a new epoch in history, it is ours to master.

What I’d like to do here is give you an overview of how I think United States policymakers should respond, as comprehensively as I can, and as I see things from my limited vantage point. I will have to cover a great deal of ground in a Substack-length essay, so many issues will be covered only at the highest levels. I will link to more in-depth resources (by me or others) where I am aware of them.

The Federal Role in AI Safety

We do not yet know enough about AI catastrophic risk to pass regulations such as top-down controls on AI models. Setting aside whether any such regime would ever really work (I have written consistently that it will not, and my confidence in that belief is higher than ever given recent developments in the field), it would entail many high costs in terms of innovation, political economy, and open scientific inquiry.

Major AI risks, and issues such as AI alignment, are primarily scientific and engineering, rather than regulatory, problems. A great deal of AI governance and risk mitigation, whether for mundane or catastrophic harms, relies upon the ability to rigorously evaluate and measure the capabilities of AI systems. Thus, the role of government should be, first and foremost, to ensure a basic standard of transparency is observed by the frontier labs.

Before I can get to that, though, there are some mistakes I believe we would be wise to correct.

Fixing the Biden Administration’s Errors

The term “AI Safety” has been massively overburdened by the Biden administration. Rather than focusing simply on the novel and major risks AI may present, the administration chose to cram a decade’s work of technology “ethics” ideas that had been percolating among bureaucrats, advocacy organizations, NGOs, academics, consultants, and others. The result is that “AI safety” and related policies tend to be unfocused, concerned with all conceivable risks, and founded on an inherently political and ideological agenda.

A large bureaucratic apparatus has begun to emerge around this nebulous and all-encompassing conception of AI safety. This apparatus should be rethought. Specifically:

1. The Biden Executive Order on AI contains a huge range of provisions, but the reporting requirements on frontier labs, biological foundation models, and large data centers are among the most important. The GOP platform promised a repeal of the EO; if that does happen, it should be replaced with an EO that substantively preserves these requirements (though the compute threshold will need to be raised over time). The EO mostly served as a starting gun for other federal efforts, however, so repealing it on its own does little.

2. Rewrite the National Institute for Standards and Technology’s AI Risk Management Framework (RMF). The RMF in its current form is a comically overbroad document, aiming to present a fully general framework for mitigating all risks of all kinds against all people, organizations, and even “the environment.” The RMF is quickly becoming a de facto law, with state legislation imposing it as a minimum standard, and advocates urging the Federal Trade Commission to enforce it as federal law. Because the RMF advises developers and corporate users of AI to talk to take approximately every conceivable step to mitigate risk, treating the RMF as a law will result in a NEPA-esque legal regime for AI development and deployment, creating an opportunity for anyone to sue any developer or corporate user of AI for, effectively, anything. The RMF should be replaced with a far more focused document—in fact, the AISI’s 800-1 guidance, while in my view flawed, comes much closer to what is needed.

3. Revise the Office of Management and Budget’s guidance for federal agency use of AI. This document substantially overregulates the federal government’s own use of AI, which creates needless burdens to AI adoption by agencies. Pre-existing agency rules and regulations, to say nothing of a bureaucracy’s natural inclination against change, will be plenty to overcome. If anything, the federal government should follow the lead of states like Utah, which have created regulatory “sandboxes”—relaxation of existing rules in some contexts—to aid in AI adoption.

4. Withdraw from the Council of Europe Framework Convention on Artificial Intelligence. This is a treaty entered into by the Biden administration in September 2024, and commits the United States to a sweeping range of policies, some of which are the same sort of nebulous technology ethics ideas mentioned above.

5. Retract the Blueprint for an AI Bill of Rights.

Legislative Action

Preemption
State governments perceive the lack of federal action on AI not as a sign that they should exercise restraint, but as an opportunity to exert unilateral authority over AI for the entire country. This was the explicit motivation behind bills like SB 1047. The European Union has even set up a technology policy office in San Francisco and advised Sacramento legislators on several AI bills in California’s last legislative session. Other states are barreling ahead with civil rights-inflected laws that create a dense thicket of open-ended rules for both developers and corporate users of AI to follow. One of these bills is already law in Colorado, even though in his signing statement Governor Jared Polis expressed concerns about the bill’s “complex compliance regime.”

This is a national security and economic competitiveness problem. We cannot have state governments, with no visibility into the geopolitical and national security dynamics of AI development, making decisions about the trajectory of this technology for the entire country. That is not to say that no role exists for the states—only that it is a considerably more limited role than the states have tried to carve out for themselves. Preemption is therefore an urgent priority. I have authored, with Brian Chau and Van Lindberg, a detailed proposal about how to structure preemption.

Transparency
As discussed above, major AI risks are primarily a scientific and otherwise technical problem, better susceptible to evaluation and study than they are to regulation. Even more mundane aspects of AI governance—such as ensuring that AI models comply with existing regulations in industrial use cases—are, at their core, evaluation problems. If a doctor wants to know whether his use of an AI model in patient diagnostics is a medical malpractice risk, his inquiry will not be meaningfully aided by a law that says “do not use AI models irresponsibly.” Instead, he will need high-quality evaluations to gauge whether different models are appropriate for his envisioned use cases.

It is conceivable that an entire ecosystem of AI model evaluators—composed of both existing and altogether new organizations—could be a major part of the long-term solution to AI governance. These organizations could purely focus on research and measurement of model capabilities, or they could take on a role more like that of insurance—“underwriting” models for specific use cases. They could even compete with one another in a “regulatory market,” as proposed by Jack Clark and Gillian Hadfield. The optimal contours of this are, as they say, an “open research question,” which is to say, “we currently have little idea.” But needless to say, evaluation and measurement are foundational parts of nearly all AI governance.

Ensuring a high-quality information environment, therefore, is essential. This is where transparency comes into play.

My optimal transparency law would be a regulation imposed on frontier AI companies, as opposed to frontier AI models. Regulating models is a novel and quite possibly fruitless endeavor; regulating a narrow range of firms, on the other hand, is something we understand how to do. I’ve written a proposal for how a regulatory threshold that exclusively targets the largest AGI labs might work in practice and am eager for feedback. It should be published soon.

The transparency bill would require that labs publicly release the following documents:

1. Responsible scaling policies—documents outlining a company’s risk governance framework as model capabilities improve. Anthropic, OpenAI, and DeepMind already have published such documents.

2. Model Specifications—technical documents detailing the developer’s desired behavior of their models.

Unless and until the need is demonstrated, these documents would be subject to no regulatory approval of any kind. The requirement is simply that they be published, and that frontier AI companies observe the commitments made in these documents. This would be enforced by whistleblower protections, which would allow employees to report suspected violations of company policy to the federal government (which agency is a question I am currently undecided on) without fear of legal retaliation.

The Role of NIST/AISI
There is currently an active debate over the future of the US AI Safety Institute (AISI), which is housed within NIST. Some want to move the functions of AISI to the Department of Energy, while others want to eliminate it altogether. Many others simply want to fund the AISI in its current form, since at present it is purely a creation of the Biden administration with no formal Congressional authorization.

My own view is that an institution like AISI must exist, but that, for the reasons outlined above relating to the expansiveness of current NIST “risk management” guidance, its mission must be narrowly circumscribed. I oppose moving AISI into an agency like the Department of Energy out of concern that it will focus overmuch on issues like nuclear weapons risk (an AI risk that is low on my priority list). Furthermore, AISI should remain a non-regulatory agency. The political economy of a centralized AI regulator is a nightmare. Because NIST is already a non-regulatory agency whose specialty is metrology (the science of measurement), it strikes me as the logical home for AI-related technical and evaluation standards.

Regardless of the details, I view the functions of AISI as follows:

1. To create technical evaluations for major AI risks in collaboration with frontier AI companies.

2. To serve as a source of expertise for other agency evaluations of frontier AI models (for example, assisting agencies testing models using classified data in their creation of model evaluations).

3. To create voluntary guidelines and standards for Responsible Scaling Policies and Model Specifications.

4. To research and test emerging safety mitigations, such as “tamper-proof” training methods that would allow open-source models to be distributed with much lower risk of having their safety features disabled through third party finetuning.

5. To research and publish technical standards for AI model security, including protection of training datasets and model weights.

6. To assist in the creation of a (largely private sector) AI evaluation ecosystem, serving as a kind of “meta-metrologist”—creating the guidelines by which others evaluate models for a wide range of uses.

NIST/AISI’s function should not be to develop fully general “risk mitigation” frameworks for all developers and users of AI models.

There is one more set of technical standards that I think merits further inquiry, but I am less certain that it belongs within NIST, so I am going to put it in a separate section.

Technical Protocols
The internet was fundamentally enabled by—indeed, substantively is—technical protocols: the Transmission Control Protocol/Internet Protocol (TCP/IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), Domain Name System (DNS), and others. All of the protocols named but one were created by the United States federal government (the exception, HTTP, was created by Tim Berners-Lee at CERN, another government-funded entity).

These protocols of the internet constitute public goods—basic infrastructure which has proved remarkably robust over time. I believe it is possible that AI may require new protocols to be invented. What might this new wave of basic infrastructure do? Here’s a short list:

1. Facilitate validation of personhood and identity in digital environments

2. Reliably identify AI agents as such, and connect them back to the users on whose behalf they are acting

3. Enable agents to make financial transactions (this could be done in dollar-denominated stablecoins, which would be yet another boon for the US dollar)

4. Ensure auditability of agent-agent interactions

The need for and technical feasibility of these protocols is speculative. They may be largely created by the private sector, or the private sector may solve these problems in different ways. But it is an area meriting further study. I see no reason not to assign this inherently basic research to the federal agency that gave us the protocols of the internet: the Defense Advanced Research Projects Agency (DARPA).

Liability
The preemption proposal mentioned above operates in part through a federal AI liability standard, rooted in the basic American concept of personal responsibility: a rebuttable presumption of user responsibility for model misuse. This means that when someone misuses a model, the law presumes they are responsible unless they can demonstrate that the model “misbehaved” in some way that they could not have reasonably anticipated.

Over the long term, there may arise a need for automated adjudication bodies that can move some AI-related litigation out of the traditional court system. This may especially be true as more capable AI agents—models that can take action on a user’s behalf—enter the market. We have already seen a glimpse of such capabilities from numerous startups and from Anthropic’s recent Computer Use feature. I suspect we will see more soon.

Say an agent tasked to do many complex tasks throughout the day for a business or individual periodically fails. These AI-enabled adjudication bodies could resolve complaints raised by users about such failures in a way that resembles small claims civil court, and is perhaps based on agency law (a branch of the common law). Small monetary damages could be assessed by these bodies on developers if the developer’s model is found to be responsible. This would function as a kind of “alignment tax,” imposing a small but noticeable financial penalty on developers for flaws in their agents. This is an area I hope to research further in the future, but I use it as an example here of a new kind of governance institution that could emerge.

Deepfakes
Deepfakes are not “exciting” to many people who think about AGI. Yet they are among the top concerns of both the public and policymakers. I have written a report that outlines my thoughts on the matter. In short, while pre-existing common and statutory law in some states provides protections against deepfakes, in other states it does not. Therefore I propose a narrowly targeted law allowing victims of malicious deepfakes to sue the individual who knowingly distributed it for damages. There should be no avenue to sue or otherwise punish website owners such as social media platforms unless the platform refuses to take down a malicious deepfake after a user has submitted a complaint.

Hardware and the Physical World

Everything I’ve written so far focuses on software. But I would be remiss if I did not mention the physical world. This is for two reasons: first, because AI is itself dependent upon vast industrial infrastructure, and second, because AI will require America to be more permissive about innovations in the physical world. AI will help us to generate many ideas for things to build—new forms of spacecraft propulsion, new kinds of drugs, new designs for nuclear fusion/fission reactors, and much else—but on its own, it will not magically build those things for us.

For the most part, this is not my specialty, so I will only summarize my thoughts briefly:

1. Permitting reform is perhaps the most important AI policy priority for the federal government. It is critical to ensure the construction of the energy generation, data centers, semiconductor manufacturing facilities, and other physical infrastructure we will require. The current compromise on the table in Congress seems reasonable to me.

2. Maintain the existing GPU and semiconductor manufacturing equipment export controls. Regardless of the wisdom of these policies, it is likely impossible to stop China from attempting to fully indigenous their semiconductor manufacturing. Weakening the controls at this point only weakens the US.

3. Critical mineral extraction and refining could emerge as a key bottleneck to manufacturing chips, and currently these markets are largely controlled by Chinese firms. Because of the high costs associated with starting new mining and refining operations in the West, this problem is unlikely to be solved purely through market forces.

4. Keep training compute on US soil but seek to aggressively build inference compute owned by American firms across the world.

5. Outside of AI infrastructure, invest in manufacturing technologies that give the US maximal flexibility and optionality to produce a wide variety of things—the basic infrastructure of manufacturing. This includes machine tools, foundries, tool and die makers, and the like. Do this through existing defense industrial base authorities like Title III of the Defense Production Act.

6. Invest in the creation of automated labs for basic scientific research. AI will enable scientists to develop more and better ideas for potential experiments, but on its own, it will do little to accelerate the experiments themselves. For this, we will need robotic labs that allow for at-scale experimentation in parallel. This is a major step along the path to automated scientific R&D. Because materials science is (1) high-impact, (2) basic research often not incentivized by the free market, (3) often repetitive and hence automation-friendly, and (4) poses far fewer safety risks when compared to fields like biological research, it is a good starting point for this investment. I will have a proposal coming out shortly for the Federation of American Scientists with more detail on this.

Conclusion

This is not comprehensive, and it is a work in progress. But I think it sets the United States along the path to success. Some of the topics above are things I have investigated thoroughly already; others are in earlier stages. I think of this essay as a kind of research agenda for myself, so if you are interested in giving feedback, discussing, or collaborating on any of the above topics, please get in touch.